Primal sketch: Integrating structure and texture

This article proposes a generative image model, which is called “primal sketch,” following Marr’s insight and terminology. This model combines two prominent classes of generative models, namely, sparse coding model and Markov random field model, for representing geometric structures and stochastic textures respectively. Specifically, the image lattice is divided into structure domain and texture domain. The sparse coding model is used to represent image intensities on the structure domain, where edge and ridge segments are modeled by image coding functions with explicit geometric and photometric parameters. The edge and ridge segments form a sketch graph whose nodes are corners and junctions. The sketch graph is governed by a simple spatial prior model. The Markov random field model is used to summarize image intensities on the texture domain, where the texture patterns are characterized by feature statistics in the form of marginal histograms of responses from a set of linear filters. The Markov random fields in-paint the texture domain while interpolating the structure domain seamlessly. A sketch pursuit algorithm is proposed for model fitting. A number of experiments on real images are shown to demonstrate the model and the algorithm.